1. Field of the Invention
This invention relates to an interpolating operation method and apparatus for an image signal.
2. Description of the Prior Art
Techniques for photoelectrically reading out an image, which has been recorded on a photographic film, in order to obtain an image signal, carrying out appropriate image processing on the image signal, and then reproducing a visible image by use of the processed image signal have heretofore been known in various fields. In image recording and reproducing systems, in which an image signal is obtained and a visible image is reproduced from the image signal, in cases where the region of interest in the visible image is to be viewed in more detail, the region of interest is often enlarged and reproduced. In such cases, if the enlargement of the image size is carried out such that the number of the image signal components of the image signal representing the enlarged image may be identical with the number of the image signal components of the original image signal representing the original image, the sharpness of the enlarged image will be recognized to be lower than the sharpness of the original image due to the visual characteristics of persons. Therefore, if the image is merely enlarged and reproduced, an enlarged image having a high sharpness cannot be obtained, and the details of the image cannot be viewed accurately.
In order for the aforesaid problems to be eliminated, a predetermined interpolating operation may be carried out on the original image signal, which has been obtained by reading out an original image, and an interpolation image signal, which is a secondary image signal and is made up of a number of image signal components different from that of the original image signal, may thereby be formed. Specifically, in cases where an enlarged image is to be reproduced, an interpolation image signal, which is made up of a number of image signal components larger than that of the original image signal, may be formed from the interpolating operation. A visible image may then be reproduced from the interpolation image signal. In this manner, the sharpness of the enlarged image can be prevented from becoming low.
As the interpolating operation methods for carrying out interpolating operations on image signals, various methods have heretofore been proposed. Among such methods, the method using third-order spline interpolating functions is popular. With the interpolating operation method using the third-order spline interpolating functions, digital original image signal components {Zk} in each section are connected by a third-order function {fk}, and the value of fk corresponding to a position, at which an interpolation point is set, (i.e., a setting position in each section) is taken as the value of the interpolated image signal component.
The interpolating operations, which pass through the original image signal in the manner described above, can yield an image having a comparatively high sharpness. As such interpolating operations, cubic spline interpolating operations, and the like, are known. How the cubic spline interpolating operations are carried out will be described hereinbelow.
FIG. 18 is an explanatory graph showing how interpolated image signal components are obtained with a cubic spline interpolating operation from original image signal components, which are sampled with a period of an equal interval and represent sampling points (picture elements) arrayed in one direction. As illustrated in FIG. 18, the image signal components (the original image signal components), which have been detected as digital signal components from an original image and represent a series of picture elements Xkxe2x88x922, Xkxe2x88x921, Xk, Xk+1, Xk+2, . . . , are respectively represented by Zkxe2x88x922, Zkxe2x88x921, Zk, Zk+1, Zk+2, . . . A third-order spline interpolating function is set for each of sections Xkxe2x88x922xcx9cXkxe2x88x921, Xkxe2x88x921xcx9cXk, Xkxcx9cXk+1, and Xk+1xcx9cXk+2. The spline interpolating functions corresponding to the respective sections are represented by fkxe2x88x922, fkxe2x88x921, fk, fk+1, and fk+2. The interpolating functions are the third-order functions, in which the position in each section serves as a variable.
Firstly, how the interpolating operation is carried out when a point taken for interpolation (hereinbelow referred to as the interpolation point) Xp falls within the section Xkxcx9cXk+1 will be described hereinbelow. The spline interpolating function fk corresponding to the section Xkxcx9cXk+1 is represented by Formula (1) shown below.
fk(x)=Akx3+Bkx2+Ckx+Dkxe2x80x83xe2x80x83(1)
In the cubic spline interpolating operation, it is necessary that the spline interpolating function fk passes through the original sampling points (picture elements), and that the first-order differential coefficient of the spline interpolating function fk is continuous between adjacent sections. Therefore, it is necessary for Formulas (2), (3), (4), and (5) shown below to be satisfied.
fk(Xk)=Zkxe2x80x83xe2x80x83(2)
fk(Xk+1)=Zk+1xe2x80x83xe2x80x83(3)
xe2x80x83fkxe2x80x2(Xk)=fkxe2x88x921xe2x80x2(Xk)xe2x80x83xe2x80x83(4)
fkxe2x80x2(Xk+1)=fk+1xe2x80x2(Xk+1)xe2x80x83xe2x80x83(5)
In these formulas, fkxe2x80x2 represents the first-order differentiation (3Akx2+2Bkx+Ck) of the function fk.
In the strict sense, the cubic spline interpolating operation contains the continuity conditions of the second-order differential coefficient. However, with continuity conditions of the second-order differential coefficient, the operation formulas become complicated. Therefore, the cubic spline interpolating operation is popularly carried out in the form simplified in the manner described above.
Also, in the cubic spline interpolating operation, it is necessary for the first-order differential coefficient at the picture element Xk to satisfy the condition with respect to the picture elements Xkxe2x88x921 and Xk+1, which are located before and after the picture element Xk, in that the first-order differential coefficient at the picture element Xk should coincide with the gradient (Zk+1xe2x88x92Zkxe2x88x921)/(Xk+1xe2x88x92Xkxe2x88x921) of the image signal components Zkxe2x88x921 and Zk+1 representing the picture elements Xkxe2x88x921 and Xk+1. Therefore, it is necessary for Formula (6) shown below to be satisfied.
fkxe2x80x2(Xk)=(Zk+1xe2x88x92Zkxe2x88x921)/(Xk+1xe2x88x92Xkxe2x88x921)xe2x80x83xe2x80x83(6)
Also, it is necessary for the first-order differential coefficient at the picture element Xk+1 to satisfy the condition with respect to the picture elements Xk and Xk+2, which are located before and after the picture element Xk+1, in that the first-order differential coefficient at the picture element Xk+1 should coincide with the gradient (Zk+2xe2x88x92Zk)/(Xk+2xe2x88x92Xk) of the image signal components Zk and Zk+2 representing the picture elements Xk and Xk+2. Therefore, it is necessary for Formula (7) shown below to be satisfied.
fkxe2x80x2(Xk+1)=(Zk+2xe2x88x92Zk)/(Xk+2xe2x88x92Xk)xe2x80x83xe2x80x83(7)
It is herein assumed that the interval (i.e., the lattice interval) of each of sections Xkxe2x88x922xcx9cXkxe2x88x921, Xkxe2x88x921xcx9cXk, Xkxcx9cXk+1, and Xk+1xcx9cXk+2 is equal to 1, and the position of the interpolation point Xp, which is taken from the picture element Xk toward the picture element Xk+1, is represented by t (0xe2x89xa6txe2x89xa61). In such cases, from Formulas (1) through (7), the formulas shown below obtain.
fk(0)=Dk=Zk
fk(1)=Ak+Bk+Ck+Dk=Zk+1
fkxe2x80x2(0)=Ck=(Zk+1xe2x88x92Zkxe2x88x921)/2
fkxe2x80x2(1)=3Ak+2Bk+Ck=(Zk+2xe2x88x92Zk)/2
Therefore, the formulas shown below obtain.
Ak=(Zk+2xe2x88x923Zk+1+3Zkxe2x88x92Zkxe2x88x921)/2
Bk=(xe2x88x92Zk+2+4Zk+1xe2x88x925Zk+2Zkxe2x88x921)/2
xe2x80x83Ck=(Zk+1xe2x88x92Zkxe2x88x921)/2
Dk=Zk
As described above, the variable conversion of X=t is carried out, and therefore the spline interpolating function fk(X) is represented by the formula shown below.
fk(X)=fk(t)
Therefore, an interpolated image signal component Zp corresponding to the interpolation point Xp may be represented by Formula (8) shown below.
Zp=fk(t)=Akt3+Bkt2+Ckt+Dkxe2x80x83xe2x80x83(8)
Substituting the coefficients Ak, Bk, Ck, and Dk into Formula (8) yields       Z    p    =                    {                              (                                          Z                                  k                  +                  2                                            -                              3                ⁢                                  xe2x80x83                                ⁢                                  Z                                      k                    +                    1                                                              +                              3                ⁢                                  Z                  k                                            -                              Z                                  k                  -                  1                                                      )                    /          2                }            ⁢              t        3              +                  {                              (                                          -                                  Z                                      k                    +                    2                                                              +                              4                ⁢                                  Z                                      k                    +                    1                                                              -                              5                ⁢                                  Z                  k                                            +                              2                ⁢                                  Z                                      k                    -                    1                                                                        )                    /          2                }            ⁢              t        2              +                  {                              (                                          Z                                  k                  +                  1                                            -                              Z                                  k                  -                  1                                                      )                    /          2                }            ⁢      t        +          Z      k      
Arranging this formula with respect to the image signal components Zkxe2x88x921, Zk, Zk+1, and Zk+2 yields Formula (9) shown below.                               Z          p                =                                            {                                                (                                                            -                                              t                        3                                                              +                                          2                      ⁢                                              t                        2                                                              -                    t                                    )                                /                2                            }                        ⁢                          Z                              k                -                1                                              +                                    {                                                (                                                            3                      ⁢                                              t                        3                                                              -                                          5                      ⁢                                              t                        2                                                              +                    2                                    )                                /                2                            }                        ⁢                          Z              k                                +                                    {                                                (                                                                                    -                        3                                            ⁢                                              t                        3                                                              +                                          4                      ⁢                                              t                        2                                                              +                    t                                    )                                /                2                            }                        ⁢                          Z                              k                +                1                                              +                                    {                                                (                                                            t                      3                                        -                                          t                      2                                                        )                                /                2                            }                        ⁢                          Z                              k                +                2                                                                        (        9        )            
The coefficients for the image signal components Zkxe2x88x921, Zk, Zk+1, and Zk+2 are referred to as the interpolation coefficients ckxe2x88x921, ck, ck+1, and ck+2. Specifically, the interpolation coefficients ckxe2x88x921, ck, ck+1, and ck+2, which respectively correspond to the image signal components Zkxe2x88x921, Zk, Zk+1, and Zk+2 in Formula (9), may be represented by the formulas shown below.
ckxe2x88x921=(xe2x88x92t3+2t2xe2x88x92t)/2
ck=(3t3xe2x88x925t2+2)/2
ck+1=(xe2x88x923t3+4t2+t)/2
ck+2=(t3xe2x88x92t2)/2
The operations described above are repeated for the sections Xkxe2x88x922xcx9cXkxe2x88x921, Xkxe2x88x921xcx9cXk, Xkxcx9cXk+1, and Xk+1xcx9cXk+2. In this manner, an interpolation image signal can be obtained, which is made up of image signal components occurring at intervals different from those of the image signal components of the entire original image signal.
As described above, in the cubic spline interpolating operation, it is necessary that the spline interpolating function passes through the original sampling points (picture elements), and that the first-order differential coefficient of the spline interpolating function is continuous between adjacent sections. With the interpolating function for the cubic spline interpolating operation, the interpolation image signal for use in the reproduction of a secondary image (i.e., the image obtained from the interpolating operation), which has a comparatively high sharpness, is obtained. On the other hand, as for a portion in the original image, at which the change in density is gentle, the interpolating operation should preferably be carried out such that a secondary image, in which the sharpness is comparatively low and which is smooth, may be obtained. As the interpolating function for obtaining the interpolation image signal representing the secondary image, in which the sharpness is comparatively low and which is smooth, for example, a B spline interpolating operation function is known. In the B spline interpolating operation, the spline interpolating function need not pass through the original sampling points (picture elements), and it is necessary that the first-order differential coefficient and the second-order differential coefficient {represented by fxe2x80x3 (X)} of the spline interpolating function are continuous between adjacent sections.
Specifically, in Formula (1),
fk(x)=Akx3+Bkx2+Ckx+Dkxe2x80x83xe2x80x83(1)
the conditions shown below should be satisfied.
fkxe2x80x2(Xk)=fkxe2x88x921xe2x80x2(Xk)xe2x80x83xe2x80x83(4)
fkxe2x80x2(Xk+1)=fk+1xe2x80x2(Xk+1)xe2x80x83xe2x80x83(5)
fkxe2x80x3(Xk)=fkxe2x88x921xe2x80x3(Xk)xe2x80x83xe2x80x83(10)
fkxe2x80x3(Xk+1)=fk+1xe2x80x3(Xkxe2x88x921)xe2x80x83xe2x80x83(11)
Also, it is necessary for the first-order differential coefficient at the picture element Xk to satisfy the condition with respect to the picture elements Xkxe2x88x921 and Xk+1, which are located before and after the picture element Xk, in that the first-order differential coefficient at the picture element Xk should coincide with the gradient (Zk+1xe2x88x92Zkxe2x88x921)/(Xk+1xe2x88x92Xkxe2x88x921) of the image signal components Zkxe2x88x921 and Zk+1 representing the picture elements Xkxe2x88x921 and Xk+1. Therefore, it is necessary for Formula (6) shown below to be satisfied.
fkxe2x80x2(Xk)=(Zk+1xe2x88x92Zkxe2x88x921)/(Xk+1xe2x88x92Xkxe2x88x921)xe2x80x83xe2x80x83(6)
Further, it is necessary for the first-order differential coefficient at the picture element Xk+1 to satisfy the condition with respect to the picture elements Xk and Xk+2, which are located before and after the picture element Xk+1, in that the first-order differential coefficient at the picture element Xk+1 should coincide with the gradient (Zk+2xe2x88x92Zk)/(Xk+2xe2x88x92Xk) of the image signal components Zk and Zk+2 representing the picture elements Xk and Xk+2. Therefore, it is necessary for Formula (7) shown below to be satisfied.
fkxe2x80x2(Xk+1)=(Zk+2xe2x88x92Zk)/(Xk+2xe2x88x92Xk)xe2x80x83xe2x80x83(7)
In general, the function f(X) may be approximately represented by Formula (12) shown below.
f(X)=f(0)+fxe2x80x2(0)X+{fxe2x80x3(0)/2}X2xe2x80x83xe2x80x83(12)
It is herein assumed that the interval (i.e., the lattice interval) of each of sections Xkxe2x88x922xcx9cXkxe2x88x921, Xkxe2x88x921xcx9cXk, Xkxcx9cXk+1, and Xk+1xcx9cXk+2 is equal to 1, and the position of the interpolation point Xp, which is taken from the picture element Xk toward the picture element Xk+1, is represented by t (0xe2x89xa6txe2x89xa61). In such cases, from Formulas (1), (4), (5), (6), (7), (10), (11), and (12), the formulas shown below obtain.
fkxe2x80x2(0)=Ck=(Zk+1xe2x88x92Zkxe2x88x921)/2
fkxe2x80x2(1)=3Ak+2Bk+Ck=(Zk+2xe2x88x92Zk)/2
fkxe2x80x3(0)=Zk+1xe2x88x922Zk+Zkxe2x88x921=2B
Therefore, the formulas shown below obtain.
Ak=(Zk+2xe2x88x923Zk+1+3Zkxe2x88x92Zkxe2x88x921)/6
Bk=(Zk+1xe2x88x922Zk+Zkxe2x88x921)/2
Ck=(Zk+1xe2x88x92Zkxe2x88x921)/2
Since Dk is unknown, it is represented by the formula
Dk=(D1Zk+2+D2Zk+1+D3Zk+D4Zkxe2x88x921)/6
As described above, the variable conversion of X=t is carried out, and therefore the spline interpolating function fk(x) is represented by the formula shown below.
fk(x)=fk(t)
Therefore,             f      k        ⁢          (      t      )        =                    {                              (                                          Z                                  k                  +                  2                                            -                              3                ⁢                                  Z                                      k                    +                    1                                                              +                              3                ⁢                                  Z                  k                                            -                              Z                                  k                  -                  1                                                      )                    /          6                }            ⁢              t        3              +                  {                              (                                          Z                                  k                  +                  1                                            -                              2                ⁢                                  Z                  k                                            +                              Z                                  k                  -                  1                                                      )                    /          2                }            ⁢              t        2              +                  {                              (                                          Z                                  k                  +                  1                                            -                              Z                                  k                  -                  1                                                      )                    /          2                }            ⁢      t        +                  (                                            D              1                        ⁢                          Z                              k                +                2                                              +                                    D              2                        ⁢                          Z                              k                +                1                                              +                                    D              3                        ⁢                          Z              k                                +                                    D              4                        ⁢                          Z                              k                -                1                                                    )            /      6      
Arranging this formula with respect to the image signal components Zkxe2x88x921, Zk, Zk+1, and Zk+2 yields Formula (13) shown below.                                           f            k                    ⁡                      (            t            )                          =                                            {                                                (                                                            -                                              t                        3                                                              +                                          3                      ⁢                                              t                        2                                                              -                                          3                      ⁢                      t                                        +                                          D                      4                                                        )                                /                6                            }                        ⁢                          Z                              k                -                1                                              +                                    {                                                (                                                            3                      ⁢                                              t                        3                                                              -                                          6                      ⁢                                              t                        2                                                              +                                          D                      3                                                        )                                /                6                            }                        ⁢                          Z              k                                +                                    {                                                (                                                                                    -                        3                                            ⁢                                              t                        3                                                              +                                          3                      ⁢                                              t                        2                                                              +                                          3                      ⁢                      t                                        +                                          D                      2                                                        )                                /                6                            }                        ⁢                          Z                              k                +                1                                              +                                    {                                                (                                                            t                      3                                        +                                          D                      1                                                        )                                /                6                            }                        ⁢                          Z                              k                +                2                                                                        (        13        )            
If t is set to be t=1, the formula shown below will obtain.             f      k        ⁢          (      1      )        =                    {                              (                                          D                4                            -              1                        )                    /          6                }            ⁢              Z                  k          -          1                      +                  {                              (                                          D                3                            -              3                        )                    /          6                }            ⁢              Z        k              +                  {                              (                                          D                2                            +              3                        )                    /          6                }            ⁢              Z                  k          +          1                      +                  {                              (                                          D                1                            +              1                        )                    /          6                }            ⁢              Z                  k          +          2                    
As for the section Xk+1xcx9cXk+2, as in Formula (13), Formula (14) shown below obtains.                                           f                          k              +              1                                ⁡                      (            t            )                          =                                            {                                                (                                                            -                                              t                        3                                                              +                                          3                      ⁢                                              t                        2                                                              -                                          3                      ⁢                      t                                        +                                          D                      4                                                        )                                /                6                            }                        ⁢                          Z              k                                +                                    {                                                (                                                            3                      ⁢                                              t                        3                                                              -                                          6                      ⁢                                              t                        2                                                              +                                          D                      3                                                        )                                /                6                            }                        ⁢                          Z                              k                +                1                                              +                                    {                                                (                                                                                    -                        3                                            ⁢                                              t                        3                                                              +                                          3                      ⁢                                              t                        2                                                              +                                          3                      ⁢                      t                                        +                                          D                      2                                                        )                                /                6                            }                        ⁢                          Z                              k                +                2                                              +                                    {                                                (                                                            t                      3                                        +                                          D                      1                                                        )                                /                6                            }                        ⁢                          Z                              k                +                3                                                                        (        14        )            
If t is set to be t=0, the formula shown below will obtain.             f              k        +        1              ⁢          (      0      )        =                    (                              D            4                    /          6                )            ⁢              Z        k              +                  (                              D            3                    /          6                )            ⁢              Z                  k          +          1                      +                  (                              D            2                    /          6                )            ⁢              Z                  k          +          2                      +                  (                              D            1                    /          6                )            ⁢              Z                  k          +          3                    
From the continuity condition {fk(1)=fk+1(0)} and the condition in that the coefficients corresponding to the respective original image signal components are equal to each other, D4xe2x88x921=0, D3xe2x88x923=D4, D2+3=D3, D1+1=D2, and D1=0. Therefore,
Dk=(Zk+1+4Zk+Zkxe2x88x921)/6
Accordingly, Formula (15) shown below obtains.                               Z          p                =                                            f              k                        ⁡                          (              t              )                                =                                                    {                                                      (                                                                  -                                                  t                          3                                                                    +                                              3                        ⁢                                                  t                          2                                                                    -                                              3                        ⁢                        t                                            +                      1                                        )                                    /                  6                                }                            ⁢                              Z                                  k                  -                  1                                                      +                                          {                                                      (                                                                  3                        ⁢                                                  t                          3                                                                    -                                              6                        ⁢                                                  t                          2                                                                    +                      4                                        )                                    /                  6                                }                            ⁢                              Z                k                                      +                                          {                                                      (                                                                                            -                          3                                                ⁢                                                  t                          3                                                                    +                                              3                        ⁢                                                  t                          2                                                                    +                                              3                        ⁢                        t                                            +                      1                                        )                                    /                  6                                }                            ⁢                              Z                                  k                  +                  1                                                      +                                          (                                                      t                    3                                    /                  6                                )                            ⁢                              Z                                  k                  +                  2                                                                                        (        15        )            
Therefore, the interpolation coefficients bkxe2x88x921, bk, bk+1, and bk+2, which respectively correspond to the image signal components Zkxe2x88x921, Zk, Zk+1, and Zk+2, may be represented by the formulas shown below.
bkxe2x88x921=(xe2x88x92t3+3t2xe2x88x923t+1)/6
bk=(3t3xe2x88x926t2+4)/6
bk+1=(xe2x88x923t3+3t2+3t+1)/6
bk+2=t3/6
The operations described above are repeated for the sections Xkxe2x88x922xcx9cXkxe2x88x921, Xkxe2x88x921xcx9cXk, Xkxcx9cXk+1, and Xk+1xcx9cXk+2. In this manner, an interpolation image signal can be obtained, which is made up of image signal components occurring at intervals different from those of the image signal components of the entire original image signal.
In this manner, in cases where a secondary image (an interpolation image) having a high sharpness is to be reproduced, for example, the cubic spline interpolating operation may be used. In cases where a secondary image, which has a low sharpness and is smooth, is to be reproduced, for example, the B spline interpolating operation may be used.
In cases where the object of an image is a person, patterns of clothes put on the person and a pattern of the face of the person may be embedded in the image. In cases where the size of such an image is to be enlarged, it is desired that the patterns of designs of the clothes can be reproduced with a high sharpness and a flesh-color region, such as the pattern of the face of the person, can be reproduced with a low sharpness, with noise, such as graininess, being removed, and to be smooth. However, in cases where the size of an image is enlarged with the cubic spline interpolating operation or the B spline interpolating operation described above, processing cannot be carried out such that the sharpness at a certain portion of the image may be enhanced or such that a certain portion of the image may be rendered smooth. Therefore, it is necessary to select whether the sharpness is to be sacrificed or the noise reducing effects are to be sacrificed, and the image size enlargement cannot be carried out such that the two requirements concerning the sharpness and noise may be satisfied at the same time.
The primary object of the present invention is to provide an interpolating operation method for an image signal, wherein image size enlargement and reduction with interpolating operations are capable of being carried out such that the sharpness at a certain portion of an image may be altered.
Another object of the present invention is to provide an interpolating operation method for an image signal, wherein an interpolation image, which is the same as an interpolation image obtained by converting color signals into a luminance signal and color difference signals and carrying out interpolating operations on the luminance signal and the color difference signals, is capable of being obtained with a reduced amount of operations.
A further object of the present invention is to provide an apparatus for carrying out the interpolating operation method for an image signal.
The present invention provides a first interpolating operation method for an image signal, wherein an interpolating operation is carried out on an original image signal, which is made up of a series of image signal components representing picture elements in an image, an interpolation image signal being obtained from the interpolating operation, the interpolation image signal being made up of a series of image signal components, which occur at intervals different from those of the image signal components of the original image signal, the method comprising the steps of:
i) carrying out interpolating operations on the original image signal and in accordance with first and second interpolating operation processes for obtaining interpolation images having different levels of sharpness, a feature measure, which represents the sharpness of the original image signal, being thereby obtained,
ii) carrying out an interpolating operation on the original image signal and in accordance with a third interpolating operation process, an interpolation image signal being thereby calculated, and
iii) correcting the interpolation image signal in accordance with the feature measure, a final interpolation image signal being thereby obtained.
In the first interpolating operation method for an image signal in accordance with the present invention, as for the first and second interpolating operation processes for obtaining interpolation images having different levels of sharpness, a B spline interpolating operation process should preferably be employed as the interpolating operation process corresponding to an image having a comparatively low sharpness, and a cubic spline interpolating operation process should preferably be employed as the interpolating operation process corresponding to an image having a sharpness higher than the comparatively low sharpness. This is because, in cases where the B spline interpolating operation process and the cubic spline interpolating operation process are combined with each other, the first-order differential coefficient becomes continuous.
However, the first interpolating operation method for an image signal in accordance with the present invention is not limited to the combination of the B spline interpolating operation process and the cubic spline interpolating operation process. For example, various interpolating operation processes, such as the B spline interpolating operation process, the cubic spline interpolating operation process, a linear interpolating operation process, and a Lagrangean interpolating operation process, may be used, and an arbitrary combination of two of these processes may be employed.
In order for the feature measure and the interpolation image signal to be calculated, they need not necessarily be obtained as the results of calculations with arithmetic formulas. For example, in cases where the interpolating operation method for an image signal in accordance with the present invention is carried out with a computer, the term xe2x80x9ccalculations of a feature measure and an interpolation image signalxe2x80x9d as used herein also means physical calculations, such as the writing of information, which represents the calculated feature measure, and information, which represents the calculated interpolation image signal, into a memory of the computer.
As the third interpolating operation process, one of the above-enumerated interpolating operation processes may be employed. Particularly, either one of the first interpolating operation process and the second interpolating operation process should preferably be employed as the third interpolating operation process.
Also, the calculation of the feature measure should preferably be carried out by calculating difference values between an interpolation image signal, which is obtained by carrying out the interpolating operation on the original image signal and in accordance with the first interpolating operation process, and an interpolation image signal, which is obtained by carrying out the interpolating operation on the original image signal and in accordance with the second interpolating operation process.
Alternatively, the calculation of the feature measure may be carried out by calculating difference interpolation coefficients, which represent differences between interpolation coefficients in the first interpolating operation process and interpolation coefficients in the second interpolating operation process, carrying out an interpolating operation on the original image signal and in accordance with the difference interpolation coefficients, and thereby calculating a difference interpolation image signal.
In a second interpolating operation method for an image signal in accordance with the present invention, the first interpolating operation method for an image signal in accordance with the present invention is applied particularly to a color image. Specifically, the present invention also provides a second interpolating operation method for an image signal, wherein an interpolating operation is carried out on an original image signal, which is made up of a series of image signal components representing picture elements in a color image, an interpolation image signal being obtained from the interpolating operation, the interpolation image signal being made up of a series of image signal components, which occur at intervals different from those of the image signal components of the original image signal, the method comprising the steps of:
i) converting R, G, and B color signal components, which represent each of the picture elements in the color image represented by the original image signal, into a luminance signal component and a color difference signal component, which represent each of the picture elements in the color image,
ii) calculating an interpolated luminance signal component by employing the first interpolating operation method for an image signal in accordance with the present invention with respect to the thus obtained luminance signal components, which represent the picture elements in the color image,
iii) calculating an interpolated color difference signal component by employing an interpolating operation process, which attaches importance to stability, with respect to the thus obtained color difference signal components, which represent the picture elements in the color image, and
iv) converting the interpolated luminance signal component and the interpolated color difference signal component into R, G, and B color signal components,
an interpolation image signal, which is composed of the R, G, and B color signal components having been obtained from the conversion of the interpolated luminance signal component and the interpolated color difference signal component, being thereby obtained.
The interpolating operation process, which attaches importance to stability, is the one which ensures the continuity of the signal values when an image signal component is interpolated from the original signal components. By way of example, the interpolating operation process, which attaches importance to stability, may be the B spline interpolating operation process described above.
The present invention further provides a first interpolating operation apparatus for an image signal, wherein an interpolating operation is carried out on an original image signal, which is made up of a series of image signal components representing picture elements in an image, an interpolation image signal being obtained from the interpolating operation, the interpolation image signal being made up of a series of image signal components, which occur at intervals different from those of the image signal components of the original image signal, the apparatus comprising:
i) a feature measure calculating means for carrying out interpolating operations on the original image signal and in accordance with first and second interpolating operation processes for obtaining interpolation images having different levels of sharpness, a feature measure, which represents the sharpness of the original image signal, being thereby obtained,
ii) an interpolating operation means for carrying out an interpolating operation on the original image signal and in accordance with a third interpolating operation process, an interpolation image signal being thereby calculated, and
iii) a correction means for correcting the interpolation image signal in accordance with the feature measure, a final interpolation image signal being thereby obtained.
In the first interpolating operation apparatus for an image signal in accordance with the present invention, as for the first and second interpolating operation processes for obtaining interpolation images having different levels of sharpness, the B spline interpolating operation process should preferably be employed as the interpolating operation process corresponding to an image having a comparatively low sharpness, and the cubic spline interpolating operation process should preferably be employed as the interpolating operation process corresponding to an image having a sharpness higher than the comparatively low sharpness. However, the first interpolating operation apparatus for an image signal in accordance with the present invention is not limited to the combination of the B spline interpolating operation process and the cubic spline interpolating operation process. For example, various other interpolating operation processes, such as the linear interpolating operation process and the Lagrangean interpolating operation process, may be used.
Also, in the first interpolating operation apparatus for an image signal in accordance with the present invention, as the third interpolating operation process, either one of the first interpolating operation process and the second interpolating operation process should preferably be employed.
Further, the feature measure calculating means should preferably comprise means for calculating, as the feature measure, difference values between an interpolation image signal, which is obtained by carrying out the interpolating operation on the original image signal and in accordance with the first interpolating operation process, and an interpolation image signal, which is obtained by carrying out the interpolating operation on the original image signal and in accordance with the second interpolating operation process.
Alternatively, the feature measure calculating means may comprise:
means for calculating difference interpolation coefficients, which represent differences between interpolation coefficients in the first interpolating operation process and interpolation coefficients in the second interpolating operation process, and
means for calculating, as the feature measure, a difference interpolation image signal, which is obtained by carrying out an interpolating operation on the original image signal and in accordance with the difference interpolation coefficients.
In a second interpolating operation apparatus for an image signal in accordance with the present invention, the first interpolating operation method for an image signal in accordance with the present invention is applied particularly to a color image. Specifically, the present invention still further provides a second interpolating operation apparatus for an image signal, wherein an interpolating operation is carried out on an original image signal, which is made up of a series of image signal components representing picture elements in a color image, an interpolation image signal being obtained from the interpolating operation, the interpolation image signal being made up of a series of image signal components, which occur at intervals different from those of the image signal components of the original image signal, the apparatus comprising:
i) a first conversion means for converting R, G, and B color signal components, which represent each of the picture elements in the color image represented by the original image signal, into a luminance signal component and a color difference signal component, which represent each of the picture elements in the color image,
ii) an interpolated luminance signal component calculating means for calculating an interpolated luminance signal component by employing the first interpolating operation method for an image signal in accordance with the present invention with respect to the thus obtained luminance signal components, which represent the picture elements in the color image,
iii) an interpolated color difference signal component calculating means for calculating an interpolated color difference signal component by employing an interpolating operation process, which attaches importance to stability, with respect to the thus obtained color difference signal components, which represent the picture elements in the color image, and
iv) a second conversion means for converting the interpolated luminance signal component and the interpolated color difference signal component into R, G, and B color signal components,
an interpolation image signal, which is composed of the R, G, and B color signal components having been obtained from the conversion of the interpolated luminance signal component and the interpolated color difference signal component, being thereby obtained.
The present invention also provides a third interpolating operation method for an image signal, wherein an interpolating operation is carried out on an original image signal, which is made up of a series of image signal components representing picture elements in a color image, an interpolation image signal being obtained from the interpolating operation, the interpolation image signal being made up of a series of image signal components, which occur at intervals different from those of the image signal components of the original image signal, the method comprising the steps of:
i) carrying out an interpolating operation on each of R color signal components, G color signal components, and B color signal components, which represent the picture elements in the color image represented by the original image signal, the interpolating operation being carried out by employing a predetermined interpolating operation process, intermediate interpolated color signal components being thereby calculated with respect to each of the R color signal components, the G color signal components, and the B color signal components,
ii) calculating luminance signal components of the original image signal from the R, G, and B color signal components, which represent the picture elements in the color image represented by the original image signal,
iii) calculating a feature measure, which represents the sharpness of the original image signal, from the luminance signal components, and
iv) correcting the intermediate interpolated color signal components in accordance with the feature measure,
an interpolation image signal, which is composed of the corrected R, G, and B intermediate interpolated color signal components, being thereby obtained.
In the third interpolating operation method for an image signal in accordance with the present invention, the predetermined interpolating operation process should preferably be an interpolating operation process, which attaches importance to stability.
Also, the calculation of the feature measure should preferably be carried out by calculating difference interpolation coefficients, which represent differences between interpolation coefficients in an interpolating operation process, which attaches importance to sharpness, and interpolation coefficients in the predetermined interpolating operation process, carrying out an interpolating operation on the luminance signal components and in accordance with the difference interpolation coefficients, and thereby calculating difference interpolated luminance signal components.
The interpolating operation process, which attaches importance to stability, is the one which ensures the continuity of the signal values when an image signal component is interpolated from the original signal components. By way of example, the interpolating operation process, which attaches importance to stability, may be the B spline interpolating operation process described above. The interpolating operation process, which attaches importance to sharpness, is the one in which the sharpness is emphasized. By way of example, the interpolating operation process, which attaches importance to sharpness, may be the cubic spline interpolating operation process described above.
Further, in the third interpolating operation method for an image signal in accordance with the present invention, the calculation of the feature measure should preferably be carried out by extracting edge components from the luminance signal components, and carrying out an interpolating operation on the edge components and in accordance with an interpolating operation process, which attaches importance to sharpness.
The present invention further provides a third interpolating operation apparatus for an image signal, wherein an interpolating operation is carried out on an original image signal, which is made up of a series of image signal components representing picture elements in a color image, an interpolation image signal being obtained from the interpolating operation, the interpolation image signal being made up of a series of image signal components, which occur at intervals different from those of the image signal components of the original image signal, the apparatus comprising:
i) an intermediate interpolating operation means for carrying out an interpolating operation on each of R color signal components, G color signal components, and B color signal components, which represent the picture elements in the color image represented by the original image signal, the interpolating operation being carried out by employing a predetermined interpolating operation process, intermediate interpolated color signal components being thereby calculated with respect to each of the R color signal components, the G color signal components, and the B color signal components,
ii) a luminance signal component calculating means for calculating luminance signal components of the original image signal from the R, G, and B color signal components, which represent the picture elements in the color image represented by the original image signal,
iii) a feature measure calculating means for calculating a feature measure, which represents the sharpness of the original image signal, from the luminance signal components, and
iv) a correction means for correcting the intermediate interpolated color signal components in accordance with the feature measure,
an interpolation image signal, which is composed of the corrected R, G, and B intermediate interpolated color signal components, being thereby obtained.
In the third interpolating operation apparatus for an image signal in accordance with the present invention, the predetermined interpolating operation process should preferably be an interpolating operation process, which attaches importance to stability.
Also, the feature measure calculating means should preferably be means that carries out the calculation of the feature measure by calculating difference interpolation coefficients, which represent differences between interpolation coefficients in an interpolating operation process, which attaches importance to sharpness, and interpolation coefficients in the predetermined interpolating operation process, carrying out an interpolating operation on the luminance signal components and in accordance with the difference interpolation coefficients, and thereby calculating difference interpolated luminance signal components.
Further, in the third interpolating operation apparatus for an image signal in accordance with the present invention, the feature measure calculating means should preferably be means that carries out the calculation of the feature measure by extracting edge components from the luminance signal components, and carrying out an interpolating operation on the edge components and in accordance with an interpolating operation process, which attaches importance to sharpness.
With the first interpolating operation method and apparatus for an image signal in accordance with the present invention, the interpolating operations are carried out on the original image signal and in accordance with the first and second interpolating operation processes for obtaining interpolation images having different levels of sharpness, and the feature measure, which represents the sharpness of the original image signal, is thereby obtained. Also, the interpolating operation is carried out on the original image signal and in accordance with the third interpolating operation process, and the interpolation image signal is thereby calculated. The interpolation image signal is then corrected in accordance with the feature measure, and the final interpolation image signal is thereby obtained. Since the interpolation image signal is thus corrected in accordance with the feature measure, which represents the sharpness of the image, the sharpness can be emphasized with respect to an image portion, which is to be reproduced with a high level of sharpness. Also, the sharpness can be reduced with respect to an image portion, which is to be reproduced to be smooth with a low level of sharpness. Therefore, the sharpness at a certain portion of the image can be enhanced, or a certain portion of the image can be rendered smooth, such that the sharpness or the noise reducing effects may not be sacrificed.
With the second interpolating operation method and apparatus for an image signal in accordance with the present invention, the same effects as those described above can be obtained.
With the third interpolating operation method and apparatus for an image signal in accordance with the present invention, the interpolating operation is carried out on each of the R color signal components, G color signal components, and B color signal components, which represent the picture elements in the color image represented by the original image signal, the interpolating operation being carried out by employing the predetermined interpolating operation process. The intermediate interpolated color signal components are thereby calculated with respect to each of the R color signal components, the G color signal components, and the B color signal components. Also, the luminance signal components of the original image signal are calculated from the R, G, and B color signal components, which represent the picture elements in the color image represented by the original image signal. Further, the feature measure, which represents the sharpness of the original image signal, is calculated from the luminance signal components. The intermediate interpolated color signal components are then corrected in accordance with the feature measure, and the final interpolation image signal, which is composed of the corrected R, G, and B intermediate interpolated color signal components, is thereby obtained. Since the intermediate interpolated color signal components are thus corrected in accordance with the feature measure, which represents the sharpness of the original image, the sharpness can be emphasized with respect to an image portion, which is to be reproduced with a high level of sharpness. Also, the sharpness can be reduced with respect to an image portion, which is to be reproduced to be smooth with a low level of sharpness. Therefore, the sharpness at a certain portion of the image can be enhanced, or a certain portion of the image can be rendered smooth, such that the sharpness or the noise reducing effects may not be sacrificed. Further, with the third interpolating operation method and apparatus for an image signal in accordance with the present invention, wherein only the luminance signal components are calculated from the original image signal, it is not necessary for operations to be carried out for completely converting the original image signal into the luminance signal components and color difference signal components. Accordingly, the time required for operations can be kept short, and the interpolating operation can be carried out quickly.